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ABSTRACT 

A study investigated the feasibility of teaching 
conceptual complexity to adults using an in-basket simulation* 
Training incorporated Kelly ! s components of differentiation and 
integration with Schroder ! s Cognitive Competencies and followed 
Lewin's Experiential Learning Model ♦ Research participants in the 
original study were 24 women and 18 men, aged 25-55, attending an 
experienced learner baccalaureate program. Research participants in 
the replication study, were 10 women and 10 men enrolled in management 
development courses offered in a community college management 
development program and/or an external Master ! s in management 
program. The instrument consisted of a 41-page, self-paced workbook 
that included pretest, text, in-basket, feedback on pretest and 
in-basket exercises, and posttest. The overall training effect was 
significant, confirming the research hypothesis that conceptual 
complexity could be improved through explicit training. In addition, 
in the original study, women scored significantly higher than men and 
showed greater improvement from pretest to posttest. (Appendixes 
contain a list of 57 references, 3 data tables, and 1 figure.) 
(Author/YLB) 
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Abstract 

We investigated the feasibility of teaching conceptual complexity to adults 
using an in-basket simulation. Training incorporated Kelly's (1955) components of 
differentiation and integration with Schroder's (1986; 1989) Cognitive Competencies 
and followed Lewin's Experiential Learning Model (Kolb, 1984). Research 
participants in the original study were 24 women and 18 men, ages 25-55, attending 
an experienced learner baccalaureate program. Research participants in the 
replication study were 10 women and 10 men enrolled in management development 
courses offered from a community college management development program and/or 
an external M.S.-Managemcnt program. The author-developed instrument consisted 
of a 41-page self-paced workbook that included pretest, text, in-basket, feedback on 
pretest and in-basket exercises, and posttest. The overall training effect was 
significant, confirming the research hypothesis that conceptual complexity can be 
improved through explicit training. In addition, in the original study, women scored 
significantly higher than men and showed greater improvement from pretest to 
posttest. 
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"The explanations J for the failure of many American corporations to adapt to 
changed economic circumstances) are fundamentally psychological, significantly 
having to do with.. ..inadequate management of change and inability to recognize and 
manage cognitive complexity..." (Levinson, 1994, p. 428). 

Today's organizations require leaders able to cope effectively with constant 
change. Conceptual complexity in making decisions is a requisite leadership skill in 
an uncertain environment (Levinson, 1994), when the repercussions of some actions 
may not be known in advance, and when some outcomes may potentially be as bad as 
others are good. The primary purpose of this study w as to explore the feasibility of 
teaching conceptual complexity using an in-basket training format. 

Conceptual Complexity 

Conceptual complexity/simplicity (Schroder, Driver, & Streufert, 1967; 
Streufert & Swezey, 1986) refers to the degree to which an individual both 
discriminates among and integrates multidimensional information. The continuum 
runs from unidimensional information processing and ensuing decision-making 
(conceptual simplicity), on the extreme low end of the conceptual complexity scale, to 
multidimensional information processing and resulting decision-making (conceptual 
complexity), on the extreme high end of the scale. The focus of the construct is on 
cognitive processes rather than outcomes. People with higher levels of conceptual 
complexity are able to perceive more dimensions and to make sharper distinctions in 
their environments. Additionally, they are better able than their conceptually 
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simplistic counterparts to synthesize these perceptions. 

The terms cognitive complexity, conceptual complexity, and integrative 
complexity are often used interchangeably in the literature. Kelly (1955), who coined 
the term "cognitive complexity/ 1 is recognized as the pioneer researcher into 
cognitive differentiation and integration, followed by Bieri (1955; 1966). However, 
Schroder et al. (1967), who coined the term "conceptual complexity," are cited most 
frequently in the literature, probably because of their development of the Paragraph 
Completion Test (PCT) to measure this phenomenon. For the sake of uniformity and 
conformity to the issue of complex conceptual search, differentiation, and integration, 
the term conceptual complexity was adopted for this study, which is based on 
Schroder's (1989) research on the cognitive competencies comprising conceptual 
complexity (Information Search, Concept Formation, and Conceptual Flexibility). 

If conceptual complexity is a learned response elicited by external stimuli, it 
should be possible to design training that develops and/or enhances the individual's 
capacity for conceptual complexity. Such training could become an important 
component of leadership development programs. 
Measurement of Conceptual Complexity 

The instrument developed for the study used an in-basket format to elicit 
responses consistent w ith managerial behaviors in the face of various organizational 
problems or dilemmas. This represents a major departure from such traditional 
instruments as the Paragraph Completion Test (Schroder, et al., 1967) or Bieri, et 
al.'s (1966) Role C oncept Repertoire (REP test). 
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Measurement differences--and the concurrent question about whether any of 
the researchers (including the authors of the present study) are even examining the 
same construct-may stem from what appears to be tlu> most significant flaw in much 
of the research. From review, there appears a promiscuous use of the term 
"conceptual complexity" to label a variety of cognitive phenomena. Because 
"conceptual complexity" is often used synonymously with other terms that include or 
are restricted to limited components of conceptual complexity, we cannot be sure 
these authors were all looking at the same cognitive phenomenon. Since even 
different measures across studies produced some of the same results, the differences in 
operationalization of the construct conceptual complexity may be semantic, and as 
such, not worthy of dissecting. It is possible that the different tests measure diverse 
elements of conceptual complexity (hat converge to yield common cognitive results 
(Streufert & Nogami, 1988). 

The present study avoided this definitional limitation by using "conceptual 
complexity" to describe the behaviors manifested in Schroder's ( 1989) cognitive 
competencies and measuring the differentiation and integration components Kelly 
(1955) proposed, in addition, the test and training instrument developed for the 
present study was a variant of the in-basket Schroder has used in his assessment of 
ihese competencies. Cliorvat's (1994) research conclusively demonstrated the 
construct validity of the leadership dimensions in the cognitive competencies 
measured by, among other instruments, the in-basket. Because the cognitive 
competencies can be seen as separate, independently measurable components of 
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conceptual complexity, in the present study they were trained simultaneously, but 
measured independently. 

Conceptual Complexity and Effective Leadership 

One of the earliest published reports of an empirical investigation into the 
relationship between conceptual complexity and task performance— relevant to 
effective leadership— was conducted by Kaiiins* Coffman, Lamm, and Schroder 
(1967). Kaiiins, et al. administered the Navy Tactical Simulation to 24 participants, 
matched on age and intelligence, divided into two groups, 12 integrative!) simple and 
12 integratively complex (measured by the Paragraph Completion Test) individuals. 
Each participant assumed the role of an officer w ith multiple responsibilities, leading 
a task force into enemy-held territory. Participants all began the task with no 
information about the components of their ow n forces or those of the enemy and 
could only gain needed information by asking questions of interviewers during a 
45-minute interview. 

With its record of all questions asked in the interview period, the Navy 
Tactical Simulation provided quantifiable data on information search - one 
component of conceptual complexity (Schroder, 1986). Although one goal of 
information search could be the reduction of uncertainty and diversity (cognitively 
simple goals), the cognitively complex participants typically exhibited more 
information search than did the integrative!) simple participants, karlins et al. 
(1967) concluded that when a task can be learned b\ actively manipulating the 
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environment, complex participants will ask more questions, and questions of a more 
complex nature, than their cognitively simpler counterparts. 

The effect of conceptual complexity on managerial performance has been 
difficult to measure because of the difficulty in measuring how (as opposed to what) 
people think, the traditional use of task-specific assessments, and the concentration of 
assessment on the content of people's thoughts and actions. Streufert, Pogash, and 
Piasecki (1988) developed and tested a computer-assisted mechanism for assessing 
several management competencies, of which conceptual complexity received the 
principal focus. Participants (n=l 1 1) had been in middle management for at least ten 
years, and most had masters or other post baccalaureate degrees. Streufert et al. used 
a software program that concurrently simulated action-oriented events and recorded 
performance data that were later analyzed. 

One of the primary goals posted by Streufert et al. (1988) w as to provide data 
on reliability and validity of simulation methods to use for training or assessment 
purposes. Among other findings, Streufert et al. reported significant correlations 
between the integrative strategy (complexity) measures and the measures of decision 
activity and speed of action. The results, although themselves complex, support the 
validity for simulation-based measurement and training. Cognitive structure dictates 
how an individual will combine externally and internally generated information 
(Kelly, 1955). At the heart of complexity theory is ability to manage complex and 
unstructured environment: I input. Therefore, we would expect conceptual 
complexity to he of value under difficult task situations. This is particular!) relevant 
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10 leaders in the turbulent environments confronting their organizations. 

Supportive of an hypothesized relationship between conceptual complexity 
and environmental complexity are two other studies in which conceptual complexity 
was found to play a major role in the explanation of military wins (Suedfeld, Corteen 
& McCormick, 1986) and for predicting surprise military attacks (Suedfeld & Bluck, 
1988). In examining six major battles in which Robert E. Lee's forces were drastically 
outnumbered, Suedfeld et aL (1986) compared Lee's complexity scores with those of 
his opponents, relying upon archival data (letters, reports, documents, etc). They 
found that despite the odds against him (as measured by the strength in numbers of 
his own and his opponents 1 armies), Lee was successful in the three battles in which 
he was considerably more complex than his opponents. In one of his losses he was 
only slightly more complex than his opponent; in the other two his opponents were 
slightly more complex, h would appear from this study that conceptual complexity 
contributes to the success of military leaders, at least in battle. 

Suedfeld and Bluck (1988) also used archival data to analyze nine surprise 
attacks upon nations by foreign military forces that occurred between 1941 and 1983. 
These attacks were precipitated by a decline in complexity in the attacking nation, as 
evidenced by the communications between attacker and attacked. No such decline 
appeared when differences were resolved peacefully. The authors concluded that 
compromise and negotiations require the flexibility of a multiperspcctivc approach, 
absent when crises could not be resolved without bloodshed. 

The primary limitation of the Suedfeld et al. (1986) and Suedfeld and Bluck 
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(1988) studies is in their narrow application to military leaders, that precludes 
generalization to a non-combat oriented population. Second, a post-hoc analysis of 
the process, in which researchers know the outcome, could seriously influence their 
results and conclusions. However, their conclusions are provocative and worth 
exploring further among leaders on business "battlefields." The analysis of archival 
data presents a third limitation, given the general research preference for empirical 
studies. However, the written communications Suedfeld and his associates examined 
may be more reflective of how people tl^ink in real situations (as opposed to "staged" 
circumstances) where there is no sense of being evaluated on their thinking per se. 
Thus, conceptual complexity appears to be germane to the use of many interrelated 
schema for collecting, integrating, and discriminating among information about the 
social environment, when it is charged with uncertainty and conflict. 

Given the instability facing many organizations today, the abilit) to \ie\\ 
situations from multiple perspectives contributes to a tolerance for and adaptability 
to, conflict and change, it is this ability to generate and weigh alternative courses of 
action that contributes to effective adjustment to change and uncertainty, critical for 
the successful leadership of any organization. 

in looking at the overall construct validity of Schroder's (1986) High 
Performing C ompetencies, which included the cognitive competencies examined in 
the present study, Chorvat (1994) confirmatory analysis of assessment center data 
collected on over 2UU British managers offered convincing support for the construct 
validity of the leadership dimensions in these competencies. 
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Gender Differences in Conceptual Complexity 

Most of the earlier research on conceptual complexity, like that on many other 
psychological or cognitive phenomena, involved observations by men researchers of 
men participants since they have dominated organizational leadership and, until . 
recently, business college classrooms (Padgett, 1990). The few recent studies with 
both women and men revealed inconsistent findings (Charlton & Bakan, 1989; 
Koenig& Seaman, 1974; Baxter-Magolda, 1992). The identification of gender 
differences in conceptual complexity appears to have depended upon the 
measurement of the construct, so that it is uncertain if researchers were looking at the 
same cognitive phenomenon. The inconsistencies in research results suggested the 
need for further exploration of gender differences, if any. Thus, the exploration of 
gender differences in conceptual complexity was an ancillary research concern. 
Teaching Conceptual Complexity 

When process-oriented research has been conducted it has been 
predominantly on the training of cognitive skills in such content areas as reading 
(Perfetti & Curtis, 1986), w riting (Scardamalia & Bereiter, 1986), second language 
(Carroll, 1986), mathematics (Mayer, 1986), Science (Linn, 1986), social studies 
(Voss, 1986), art (Somerville & Hartley, 1986), music (Sera fine, 1986), and even 
reasoning (Nickerson, 1986). Almost all research into teaching cognitive skills 
concentrated on children; the literature is bereft of research involving adult 
participants. Further, aside from the various attempts to teach domain-specific 
cognitive skills, there is a dearth of studies into the trainabilitv of conceptual 
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complexity. 

Streufert and Nogami ( 1988) proposed that, because it appears to increase 
from young to middle age— suggesting a learning effect of sorts— training might be 
possible. However, few have attempted it. In one of the few published studies on 
teaching conceptual complexity, Gardiner (1972), working from the premise that 
racial prejudice is a byproduct of conceptual simplicity, sought to teach white high 
school students to become conceptually complex in their interracial relations. 
Students were paid to participate in one of three experimental conditions (two 
treatment, one control), to which they were randomly assigned. Participants in the 
training conditions evaluated "candidates u for the role of ombudsperson or 
conciliator/mediator. Among the candidates in each scenario, the same number of 
black and white candidates exhibited conceptual complexity and the same number of 
black and white candidates exhibited conceptual simplicity. 

Participants in both of Gardiner's (1972) training conditions were gi\cn a list 
of three conceptual dimensions on which they were to rate candidates and were 
encouraged to add dimensions they thought germane to the job. Upon completion of 
this task, participants were then instructed to evaluate two of their friends using a 
prepared form analogous to the training condition format. 

Because post t est Race- Relations scores of participants in the training 
conditions were higher than posttest scores for the comparison group. Gardiner 
(1972) concluded that training can lead to increased conceptual complexity and to a 
reduction in racial prejudice under some circumstances. 
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Gardiner's (1972) study is fraught with methodological limitations, not the least of 
which is homogeneity of sample (white teenage boys and girls). Further, reliance on 
payment to ensure candid, cooperative participation, from a group of minors whose 
only means of income is their parents or other adults for whom they perform 
minimum-wage labor placed these students in a subordinate/authority figure 
relationship that could have inhibited honest responses. Finally, it must be 
emphasized that any training effect in this study must be recognized in the context of 
interracial relations. Thus, any attempt to generalize outside the white teen 
population or to cognitixe/emotive areas different than ethic/racial prejudice would 
be imprudent. 

The In-basket as an Instrument for Teaching Cognitive Skills 
The instructional goal, enhancing conceptual complexity, falls into Gagne's 
(1985) Problem Solving classification of intellectual skills. To "teach" conceptual 
complexity, then, the design must lead people to move away from their reliance on 
intuitive conceptions to solve problems, generally based upon personally acquired 
knowledge or experience (Linn, 1986), and toward discovery, exploration, and 
application of alternative concepts. 

The Lewinian Experiential Learning Model (Kolb, 1984), in which learning 
is a cyclical four-phase process-Concrete Experience, Reflective Observation, 
Abstract Conceptualization, and Active Experimentation— formed the theer\ upon 
which the in-basket training design was based. It was hypothesized that, with an 
experiential approach that closely simulates the kinds of activities managers w ould 
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face on the job, prospective managers could be taught to engage in the behaviors 
associated with the cognitive competencies that comprise conceptual complexity. 
Thus, although the in-basket is stil! the most widely used management assessment 
tool (Gill, 1979), it was seen here as having the potential to be a valuable 
experiential learning instrument. 

The typical in-basket, which represents the concrete experience portion of 
the experiential learning model, is an organizational simulation that elicits 
responses consistent with managerial behaviors in the face of various 
organizational problems or dilemmas. Brannick, Michaels, and Baker (1989) 
demonstrated that the in-basket can be used to train certain skills. These authors 
used a workbook designed to train three dimensions (skills)— Planning and 
Organizing, Perceptiveucss, and Delegation— and the overall sum. Of these 
dimensions, Perceptiveness (operationalized as the explicit recognition of related 
in-basket items) can be loosely construed as one element of conceptual complexity. 

Upon finding significant training effects for Perceptiveness, Delegation, and 
the overall sum, Brannick et al. (1989) concluded that training can be successfully 
designed to improve in-basket performance. Three of the four problem solving 
skills Gagne (1985) described were targeted in the in-basket developed for the 
present study. Training focused on (1) concepts (information/opinion); (2) rules 
(e.g., for determining information adequacy/deficit); and (3) discriminations 
(between appropriate/inappropriate information types and/or sources). 

Method 
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This quasi-experimcnt consisted of a reversed-treatment comparison group 
mixed factorial design ( Tabachnick & Kidell, 1983), where repeated measures were 
used to look at the change in scores from pretest to posttest for women and men in 
training and comparison groups. 

The original study was conducted as part of the first author's doctoral 
dissertation; the stud} was replicated 18 months later with a second sample. These 
two studies will be referred to as original and replication, respecthei*. 
Participants 

Table 1 presents demographic data for both sets of research participants. 

Insert Table I about here 



As Table 1 reflects, the participants in the original study were somewhat older, less 
educated, and had slightly more managerial experience (in terms of number of 
years and number of subordinates) than the participants in the replication study. 
However, these differences were not statistically significant, and in an additional 
analysis (described later) that included this and subsequent samples, the data were 
pooled. 

Original study . Research participants (n=42) were students in an adult, 
non-traditional baccalaureate program, an experiential-learning-based program in 
which much of students' work is independent and self-paced. 

Twenty (12 women, 8 men) completed the training group workbook and 22 
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(12 women, 10 men) completed the comparison group workbook. 

All but one participant (98%) were w hite; 37 were between 30 and 49 year., 
of age; 37 had some college; 23 had at least one year of managerial experience, and 
14 had more than six years of managerial experience (see Table 1). 

Replication study . Research participants (n=20) were students enrolled in 
management development courses offered from a community college management 
development program and/or an external M.S. -Management program. 

Thirteen (6 women. 7 men) completed the training group workbook and 7 
(4 women, 3 men) completed the comparison group workbook. 

As Table 1 shows, all but one participant (95%) were white; 10 were under 
30 and 10 were betw een 30 and 49 years of age; all had some college, and hall* were 
college graduates; 14 had between one and six years of managerial experience. 
Instrument 

The author-developed instrument consisted of a 41-page w orkbook that 
began with a letter of instruction and a demographic data sheet. This was 
followed, in order, by (1) pretest, (2) training text, (3) in-basket, and (4) posttest. 
The in-basket developed for the present study was not specific to an\ git en field, 
and the items reflected the kinds of things managers in a variety of organizations 
encounter daily. The bound workbook was self-paced and took most participants 
approximately two hours to complete. 

The instrument was lleldtested en regie with both managerial and 
nonnianagerial research participants not included in either study. The 
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participants were instructed to progress at their own pace, as if they were taking 
part in professional training, to ask questions as they arose, and to make any notes 
or comments the> wished in I he margins. The few changes in the materials they 
recommended were incorporated in the final product, with one exception. None 
was pleased with the self-paced, individual study format. However, it appears that 
it was not the self-paced mode, but the inability to interact with the researcher that 
was somewhat off-putting. Two pointed out that "live" training might be more 
effective, to enable participants to look to the researcher as an information 
resource; however, because one of the research questions focuses on the in-basket 
as a training instrument for conceptual complexity, the self-paced format was not 
changed. 

Tests. The pretest and posttest consisted of a mixture of essay questions 
and checklist items. The total possible score range (checklist plus essa> questions) 
for each test (pretest and posttest) was 0 to 6. 

The pretest and posttest w ere combined and presented together as one test 
to a group of students (n=24) draw n from the same academic programs as those 
who participated in both the original and replication study. The small difference 
between the significant!) correlated (r=.536,/;<.01) pre- and posttest scores did 
not even approach significance (r=1.03,/;=.29). 

The checklist questions were designed to measure the Information Search 
and Conceptual Flexibility components of conceptual complexity (Schroder, 1983) 
and they provided participants opportunities to select appropriate sources for 
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information needed in decision-making or to diagnose problems or issues in order 

to seek satisfactory resolution. An example of this follows: 

You are an executive for a large newspaper widely known for liberal 
political views and controversial attitudes. As a result of the sudden 
death of one of the department heads who reports to you, there is a 
position vacancy which must be filled quickly. The Human Resources 
director gave you a list of names of candidates. The most qualified 
candidate was diagnosed with multiple sclerosis (MS) two >ears ago, 
at which time she missed two months of work. She had an excellent 
attendance record prior to this, and has missed no work since then. 
Her MS has been in remission for over eighteen months, but MS is 
unpredictable in lerms of w hen, how , and w ith w hat severity it w ill 
manifest itself. The job is a demanding one, w ith long hours and 
considerable tra\eh Although in this company executives typically 
remain in an\ one job approximately two years, the deceased 
incumbent had been there nearly three years. He was good. It w ill be 
hard to replace him. Place a Y next to the information source(s) 
below that might help you make the best decision on filling this 
position. 

_ 37. Performance evaluations of all candidates. 

38. Medical records on all other candidates. 

39. This candidate's insurance records. 
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_ 40. Consultation with company physician (or another doctor), 
_ 4 1 . Journal of Medical Psvchosociology 
_ 42. Incumbent's travel vouchers and records. 
_ 43. The candidate herself. 
The score for each question, which ranged from 0 (competency not 
demonstrated) to 1 (competency thoroughly demonstrated), was the total number 
of choices divided by the number of correct choices for that question. This scoring 
procedure was selected lo ensure each question was weighted identically, 
uninfluenced by the difference in the numbers of choices associated with the 
different checklist questions. 

Differentiation refers to the ability to recognize multiple perspectives or 
dimensions of information. Integration denotes the ability to combine these 
various pieces of information or perspectives to arrive at a larger picture. As an 
example, the following is one of the cssa> questions in the posttest. 

You ha\e just taken over the product development 
department at Dulcc Inc. Your predecessor, voted "Boss of the 
Year" for the past two years, was fired after the local newspaper ran 
a series of investigate e reports, relying on an unidentified company 
source who provided information on falsification of safet\ reports, 
internal tests, etc. The safety inspector was never involved in the 
reports. Local sales have slackened, and department morale is low. 
As the new department head, where would you go from here? » hat 
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approaches) would you take to address the problems you inherited 
in this department? 

The essay questions were transcribed, numbered, separated from the 
checklist questions, and scored (blind to the training condition) by the researcher 
and another trained assessor, independently. Because the interrater reliability of 
these observational measures was high on the essay questions on both the pretest 
(r=.994,/;<.001) and posilest (r=. 986,/;<.00l), the average of the two raters 1 scores 
on each essay question scores was used as the final essay score on that question. 
Like the checklist questions, the score range for each essay question was 0 to I, 
depending upon the number of components of conceptual complexity present in 
each response. Again, the scoring procedure ensured that each question was 
weighted identically. For assessment purposes, the essay questions were scaled as 
follows: 

1 - exhibits integration AM) differentiation AND suggests more than one 
alternative 

.75 - exhibits integration AND differentiation (but suggests no more than one 

possible course of action or solution) 
.50 - exhibits integration OK differentiation AND suggests more than one 
alternative 

.25 - exhibits one of the three bcha\iors abo\e, either differentiation, integration, 
or 

multiple alternatives, 

zo 
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0 - exhibits none of the behaviors above 

The responses of the training group participants who answered a 
manipulation cheek question at the end of the workbook indicated they recognized 
the training goal, supporting the convergent validity of the training instrument 
(Kidder & Judd, 1980). 

Training Group liooklet . The first segment of the training booklet was a 
brief description of conceptual complexity and its relevance to effective 
organizational leadership. Following this was an explanation of the four checklist 
pretest exercises, emphasizing the components of the cognitive competencies and 
the training objectives. Each possible choice in the checklist questions was 
addressed, with an explanation of what made each individual choice correct or 
incorrect. After this brief didactic section, the remainder (and bulk) of the 
training material included the in-basket itself, with the accompanying feedback 
and boosters by w hich learners proceeded through the Concrete Experience. 
Reflective Observation, Abstract ( oiiccptuali/ation, and Acti\e Experimentation 
learning phases of kolb's (1^84) elaboration of the Lew inian Experiential Learning 
Model 

The in-baskct exercise segment represents the concrete experience described 
above. Six separate items contained different issues or problems in written form 
(telephone messages, letters, and memoranda). Some required little, whereas 
others demanded a greater deuree of, inquin, diagnosis, decision, delegation, or 
action. Each of the six items had three exercise components, simulation, feedback, 
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and booster, aimed at practicing one of the cognitive competencies. 

There are typically no "right or wrong answers" in an in-basket; however, 
the page following the essay questions contained a key with the best answers to 
each item and a "booster" reminding them of pertinent questions they must 
consider before seeking information or planning alternative courses of action. This 
feedback contributes to reflection and the resultant formation of abstract ideas and 
generalizations, the second and third portions of the experiential learning model, 
considered critical elements in the learning process. Lew in (1^>51) asserted that the 
absence of suitable feedback processes accounted for much ineffectual hchaxior 
and actions in organizations. Presenting the exercises serially gave participants the 
opportunity to transfer v»hat the> learned from one to the other, satisfying the 
Active Experimentation requirement of the cxclc. 

Comparison Group Booklet . The comparison group booklet consisted of 
the same pre- and posltest and simulation (with three additional written items to 
equalize the time spent). Missing were the introductory text, feedback on pretest 
responses, and the boosters following each in-basket item in the training group 
booklet. It was the omission of the Reflective Observation and Abstract 
Conceptualization components of the cycle that rendered this a reversed treatment. 
Procedure 

Original stucU . Se\ent\ training and 70 comparison group workbooks were 
random!) distributed b> the fu st author to an c\cn number of men and women in 
15 classes in a nontraditional baccalaureate degree program for adults. Although 
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the first author leaches in this program, neither author taught these particular 
liberal arts classes or knew uii> of the students. Students were given four weeks to 
complete the materials and return them to a marked drop-off box outside the 
classrooms. 

Despite the incentive of individual test feedback, only 42 (30%) of the 
workbooks were returned. This 30% response rate, while disappointing, is 
relatively standard for mailed suneys. Although the instrument was hand- 
delivered to participants for completion on their own time, with instructions to 
return it to a designated location at their school, it could be likened to a mailed 
survey. 

Despite the low response rale, the demographic characteristics of the 
sample, described -arlicr, are rcprcsentati\ e of those of this college program's 
student population, based upon the first author's experience in and observation of 
the student population. Unfortunately, college-maintained demographic records 
were not made available to the researchers. 

Replication stmh . Fifteen training and 15 comparison group workbooks 
were randomly distributed by the first author to an even number of men and 
women in one management development class and two external M.S. -Management 
classes. They were instructed to return their materials the follow ing class period 
(one week later), and (><>% did so. Certainly, the response rate in the replication 
study was considerably belter than in the original stud). I his may he dtie, at least 
in part, to the fact that this sample, unlike the original study, consisted of students 
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taking classes taught b> the first author, Although their responses were voluntary 
and anonymous, and participation did not affect the grade, some may have 
participated out of loyalty to their professor. 

Results 

Original Study 

There were no significant differences for age, education, or managerial 
experience on the pretest or postlesl scores: however, there were significant gender 
differences. Therefore, all women's data and all men's data were pooled for 
subsequent analyses. 

Pretest, posttest, and change score means and standard deviations for 
women and men in the training and comparison groups are presented in Table 2. 



Insert Table 2 about here 



As Table 2 indicates, the comparison group actually scored higher than the 
training group on the pretest. In addition, women's pretest scores were 
significantly higher than men's scores | l<(U38) -5.007 /;<.05|, and there was a 
significant gender by group interaction |I*'(I V 38)=8.404,/;<.01|. The H-Tukey 
statistic showed that men in the training group scored significantly lower than all 
other groups |K(3,38)=5.647,/;<.01 1. 

The training group scored significantly higher on the posttest than the 
comparison group |K(l,78) r 27.01 /;<.00l|. In addition, the training group showed 
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significantly greater iinpro\ement from pretest to posttest than did the comparison 
group |F(1,78)=48.42,/><.UUI |. Scores of 80% of the training group improved from 
pretest to posttest, while 90% of the comparison group scores worsened. 

Figure 1 displays the change in scores from pretest to posttest for women 
and men in the training and comparison groups. 

Insert Figure t about here 



Overall, women both scored significantly higher on the posttest 
|F(1,78)=12.60 /;<.001| and showed greater improvement from pretest lo posttest 
than men |F(1,78)=6.10,/;<.U5| (See Figure I). 

The interaction between treatment and gender was not significant, nor was 
the interaction between ireauneut, gender, and time. 

While the main effect for time was not significant, post hoc analysis (/-test 
of correlated means) confirmed that the training group's scores increased 
significantly from pre- lo posttest |/=-3.l8, /;<.0t ). Also, the comparison group 
scored significant!) low , v on the posttest than the pretest. 
Replication Suidx 

Because there were no significant differences for age, education, or 
managerial experience nor for gender on the pretest or posttest scores, all 
participants 1 data in the replication stud) were pooled for subsequent analyses. 

Pretest, posttest, and change score means and standard delations for the 
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training and comparison groups are presented in Table 3. 

Insert Table 3 about here 



As Table 3 indicates, there was no significant difference between training 
and comparison groups 1 pretest scores. However, the training group scored 
significantly higher on the postlesl than the comparison group [K(l,39)=5.68, 
/;<.05j. In addition, the training group showed significantly greater improvement 
from pretest to posttest than did the comparison group |F(1,39)=1 1.1 1, 
/;<.01|. 

Post hoc analysis (/-lest of correlated means) confirmed that the training 
group's scores increased significantly from pre- to posttest |f=-2.6L p<.05). 
However, unlike the original study, there was no significant decrease in the 
comparison group's scores. 

As an additional means by which to evaluate tl npact of the training 
materials on posttest, the data from the original ami implication study were pooled 
with the results of the 24 students who took the combined prctcsl/posllcsl (with no 
training materials) for one final analysis. There were no significant differences in 
pretest scores across the three samples |K(2,85)=I.56,/;=.22|. However, the 
training group posttest scores w ere significantly higher than those of both the 
comparison group and the test-only group |T\2,85)=1 1.13,/;<.001|. 

Discussion 



Teaching Conceptual Complexity 26 

The focal question in the present study was to determine if an in-basket 
training strategy would expand the dimensions along which participants think. 
While the search) of research in teaching conceptual complexity to adults offered 
little empirical foundation for this project, Gardiner's (1972), Schroder's (1975), 
and Brannick et al.'s (1989) studies were suggestive. 

In the original stud), the training group's posttest scores were significantly 
higher than those of the comparison group and than their own pretest scores, 
confirming the research hspolhesis that conceptual complexity, as operationally 
defined here, would improve. Observing these same results in the replication study 
substantiates these findings. Similarly, the results of the meta-analysis, evidence 
that the results of both original and replication studies are not spurious, reinforce 
the conclusion that the cognitive competencies associated with conceptual 
complexity can be enhanced. 

As Figure i depicts, at the outset, the two groups' scores were similar in 
both studies, and, in fact, men in the training group scored significantly lower on 
the pretest than the other three groups in the original study. Still, in both studies, 
both women and men in the training group scored significantly higher on the 
posttest than did women and men in the comparison group. The training group's 
positive change from pre- to posttest was predicted. Although one also might have 
expected that the in-basket material offered the comparison group in both studies 
would produce little change or perhaps some increase, both comparison groups' 
performance noticeabh deteriorated (See Figure I). One obvious explanation for 
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this apparent anomaly could be unreliability of the instrument. However, 
correlation of the pre- and posttest scores of a fieldtest sample, comparable to the 
study participants, indicated the two tests were significantly correlated (r=-706, 
/;<.001). A mure ltkel\ explanation of the comparison group's score decline lies in 
the negative effects produced by their counterfeit training. 

Missing from the comparison group workbook were text, examples, 
feedback on pretest and in-baskel exercises, and coaching, all of w hich completed 
Lew in's experiential learning c>ck\ Particular^ critical might ha\e been the 
deliberate omission of feedback Senge (1990) described a delusion of learning from 
experience that stems from the fact that, although experience can offer the most 
effective means by which to learn, "We never directly experience the consequences 
of many of our most important decisions." The immediate feedback given the 
training group after each in-basket item satisfied Senge's criterion of experiencing 
the consequences of one's actions, and, in tlic present study, ser\ed as an error 
correction mechanism for the training group. Thus, the omission of feedback 
from the comparison group w orkbook not onh deprived them of the positive effect 
of immediately assessing their own progress, but did not counteract the ncgati\c 
effect of fatigue or boredom, either or both of which may ha\e adverse!) affected 
comparison group performance. 

To help understand how feedback both directs and energizes behavior, we 
can extend the theory of c> bernetics to the importance of inclusion of all four 
processes in the Lewinia&i Model of Experiential Learning (Kolb, 1984). "For any 
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machine subject to a varied external environment to act effectively, it is necessary 
that information concerning the results of its own action be furnished to it. ..The 
feedback of such information... enables the |person| to correct for |his/her| own 
malfunctioning or for changes in Hie environment..." (Nadler, 1977, pp. 67-69). All 
learning is based on this process of inhibiting incorrect responses, creating 
internalized programs or learning sets (hat selectively interpret new experiences 
(Harlow, 1959). As the training group seems to have demonstrated, learning 
occurs only when participants are given feedback, the opportunity to relied on 
that feedback and draw ivle\ant conclusions, and to replicate modeled behaviors. 
Gender 

Although women scored significant!) higher than men in the original study 
on both the pre- and posttest (see Table 2), there was iio such difference on the 
replication stud). Combining the data for the two studies for subsequent meta- 
analysis revealed no gender differences in pi e- or posttest scores. As discussed 
earlier, such iucousistcuc) in findings is not unusual. 

Koenig and Seaman (1974) found men to be more conceptually complex 
than women on an instrument measuring participants' assessment of conceptual 
complexity. However, these researchers interacted directly with the participants, 
and their conclusions wi-re bast'd upon women's and men's assessments of gender- 
based stimuli. In the present stud) the sell-administered instruments were not 
sensitive to gender in these ways, and the evaluation of all test questions was 
completed blind to gender. 
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Charlton and Bakan (1989), using an essay format, found that women were 
more conceptually complex than men and attributed this difference to women's 
superior verbal fluency. In the original sample in the present stud) it was on two 
objective questions thai uuiiumi outperformed men in the pretest, and on both 
objective and subjective items in the posltcsl. Because the pretest essay questions 
failed to differentiate between women and men, we could attribute the women's 
superior posttest essay scores in the original study to their response to training, 
rather than to verbal lluenc). Women in the replication study actually scored 
slightly lower than men on both the pre- and the posttest, although the difference 
was not significant. 

Although neither (Jlaser (1982) nor Dillon (1986) mentioned gender as a 
variable of concern in response to training, the notion of personal differences may 
have relevance to the different responses to training by women and men in the 
original sample in the present studs. The acquisition of cognitive skills ma\ be a 
function of other personal variables (e.g., interest, motivation, etc.) not directly 
measured in this study. 

Limitations 

The major limitation of this study la) in its small and homogeneous 
samples. Despite the small \s in both samples, however, significant effects were 
obtained. While the participants were all student volunteers, they were in fact 
more representative of a larger population of managers and organizational leaders 
than typical college groups. Many held executive or upper-management positions. 
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As Table I reflects, 74% of the participants in the original study and 70% in the 
replication study had managerial experience. Also, they were above the traditional 
age of college students (88% in the original stud) and 50% in the replication study 
were in their thirties or forties). 

implications for Further Research 

Dcwe> (1910) proposed that the most important element of cognitive 
training is the evolution of an altitude of "suspended conclusion" (p. 13) in which 
decisions are postponed until multiple strategies are used to collect both consonant 
and dissonant information for consideration, in the absence of a good foundation 
of conceptual complexity training in the literature, it w ould be wise to withhold 
judgment on the efficac> of such training until the results of the present study can 
be replicated, particularly with a population with greater variation in managerial 
or employment experience. If other studies demonstrate that conceptual 
complexity can be taught, it should be a continuing education objective for adults 
and a priman goal of leadership training. This is especially important, given the 
unstable environmental conditions inauv organizations face, where the ability to 
view situations from multiple perspectives strengthens tolerance for, and 
adaptability to, conflict and changes. 

In view of the proposition that efleethe leadership has an holistic qualit> 
that includes other impw: tain attributes, and because conceptual complexity may 
He at the core of these attributes, we could expect conceptual complexil) training 
to have a positive impact on the larger leadership domain. 
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The ln-Basket as a Training Instrument 

The results of the present study have two critical practical implications. 
First, this study's results support the argument that the skills associated with 
conceptual complexity can he taught. Second, the results suggest that the 
in-basket may be a valuable experiential learning instrument b> which to teach 
conceptual complexity. Hie sell-paced training format here can be o lie red at a 
fraction of the cost associated with trainer-facilitated instruction. W here in-basket 
training is offered with trainer-facilitated feedback, often cost-prohibitive in terms 
of dollars and time, the ability for students to receive immediate feedback on their 
actions via prepared workbook or better, \>lien developed into an interacts e 
computer program is certainly cost-elTecli\ e. 

Because of the length of time it took participants to complete the training or 
reverse-treatment materials, and in view of the lack of control over the 
experimental situation, it may be helpful to replicate this study under more 
controlled experimental conditions, perhaps l>> means of ( A I technology 

Kuturc research should also examine whether or not this tjpe of training 
generalized to improved job performance. If so, efforts should be made to identify 
those tasks and/or jobs in which such training w ould be beneficial. 
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Table 1 

Participants' Demographic Characteristics 



Original Study 
11 % 



Replication Study 

n % 



Gender 

Women 

Men 
Race 

White 

Black 
Age 

Under 30 years 

30-39 years 

40-49 years 

50 and above 
Education 

High School 

< 2 years college 
2-4 years college 



22 57% 
18 43% 

A 1 98% 
1 2% 

7% 
38% 
50% 
5% 



3 
16 
21 



3 7% 



(> 14% 



31 7 -4% 



10 50% 
10 50% 

19 95% 
I 5% 

50% 
30% 
20% 
0% 



10 
6 
4 

0 



1 5% 

2 10% 

A 20% 
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College graduate 
Unreported 
Managerial Experience 
No experience 
< 1 year experience 

I - 3 years experience 
4-6 years experience 
> 6 years experience 

Number of subordinates 
Five or less 
6-10 subordinates 

I I - 20 subordinates 
21-50 subordinates 



0 
2 



0% 
5% 



1 1 26% 

8 19% 

5 12% 

4 10% 

14 33% 

16 38% 

4 9% 
10 24% 

5 12% 



13 65% 
0 0% 



6 
0 
6 



6 
0 
0 



30% 
0% 



8 40% 
0 0% 



30% 
0% 
0% 



1 2 60% 



More than 50 



17% 



10% 
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Table 2 

Original Study Pretest, Posllesl. ami Change Score .Means and Standard Deviations for 
Women and Men in Training and Comparison Groups 

Pretest Postlest Change 

Training Group 

Women (N=I2) Mean 4.44 5.33 .89 

SI) 1.00 .32 .87 

Men(N=8) Mean 2.% 3.45 .49 

SI) 1.31 .99 1.25 

Total (N=20) Mean 3.85 4.58 .73 

SI) 1.23 1.15 1.02 

Comparison Croup 

Women (N = 12) Mean 4.31 3.78 - .53 

SI) .82 .40 

Men (N=I0) Mean 4.44 3.32 -1.12 

SI) .70 .29 .59 

Total (N=22) Mean 4.37 3.57 - .80 

SI) ."0 .03 .59 



9 

ERIC 



44 



Teaching Conceptual Complexity 44 



Table 3 

Replication Suuh Pretest. Posllesi. and Change Score Means 
and Standard Deviations for Training and Comparison Croups 



Pretest Posttest Change 



Training Croup 
(N«6) 

Mean 4.32 
SD .59 

Comparison Croup 

(N-7) 

Mean 4.7U 
SD .62 



4.95 .63 

.85 .89 

3.41 -1.29 

.67 .85 
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Figure 1 : Change from Pretest to Posttest Scores tor Women and Men in the 
Training and Comparison (..'roups 
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